Tracking of Feature Points in Dynamic Image with Classification into Objects and 3d Reconstruction by Particle Filters

A new model for tracking of feature points in dynamic image is proposed. The model is represented in a form of nonlinear state space model having state variables with positions of feature points, velocities for each object, and object labels that specify the associations between the feature points and the objects. We use particle filters with RaoBlackwellization to estimate the state of the nonlinear model. By estimating the state, we obtain the tracking result of feature points, which consists of positions of feature points and velocities for each object, as well as the classification result of the feature points into objects from the estimate of the associations. 3D reconstruction is also dealt with in this framework by introducing a camera projection model into observation equation of the state space model. Experiments using real images for 2D tracking and 3D reconstruction show the efficiency of the model and the method.